Metrosity A real-time gesture tracking environment towards a personal interactive communication system

Many people can be observed not being aware of their fellow commuters. A system that encourages personal communication between people would give opportunities for an improvement in life quality where the richness of human contact could be realized. The abundance of online dating, chat rooms and other non human communication forms point to the populous being aware of the need to communicate. This paper presents the technical realization of an artistic interactive vision project which is currently a ”work-inprogress” towards realization in public spaces. The object of the vision project is to motivate people to interact in a ‘human-to-human’ way. Imagine, e.g., a scenario with two escalators descending, facing a big wall (see Fig. 1). This is a scenario which often appears, e.g., in metro stations. Cameras will be installed on the wall facing the escalators and the video stream will be projected onto this wall. In that sense, the wall will become a big, artificial mirror of the scenario facing it. If an individual enters the scene on the top of the escalators and starts descending he/she sees ahead of him/her the whole environment reflected on a wall, including the escalator he is standing on. By automatically detecting the individual in the mirrored image, the individuals may be replaced by avatars or, e.g., speech bubbles may be inserted into the mirror-image. In that sense, a descending individual may notice that his reflection has an overlaid image. This image is following him/her as the escalator descends. A similar event may happen in the above scenario upon the other descending escalator. The vision system can detect this event and can enter into the mirror-image speech bobbles that may initiate some sort of communication between the two individuals. A major observation is that the imaging conditions are constant over time. This considerably simplifies the construction of a system that is expected to function independently of the day time. The key techniques in realizing the vision system are